1. Field
The present invention relates generally to computer graphics and, more specifically, to image-based modeling.
2. Description
Accurate portrayal of the physical world has been an important goal of computer graphics since its inception. This includes modeling the geometry and surface attributes of the object when the object is illuminated by a light source and viewed in a virtual environment from a certain vantage point. Typically, the geometry is modeled as a polygonal mesh that models the geometric extents of the object. The surface attributes, which can be modeled in a number of ways, are then applied to the geometry during rendering to graphically display the object.
The essential part of the quest for realistic graphics is proper modeling of the visual attributes of the surfaces of objects. For example, proper modeling of surfaces that have luster, reflections, such as chrome reflections, semitransparent regions, complex textures like fur, and other qualities, is needed to create realistic visual models of many real-world objects. Theoretical and empirical approaches are possible.
Historically, parametric reflectance models, such as Phong shading, have been favored since they are simple and compact. However these techniques can be inaccurate, are difficult to implement on traditional graphics hardware, and are challenging to develop. An alternative is to use image-based techniques. Such image-based techniques offer a simple method of acquiring the visual data and an accurate method for portraying the physical world. They represent discrete visual data, acquired from the physical world using a variety of imaging devices, directly in the sample-based format without resorting to the parametric models. Image-based modeling refers to sampling the shape and/or appearance of an object using a set of images collected from an imaging device (such as a digital camera, for example), and building a three-dimensional (3D) model from the acquired image-based data. A user may then interact with the 3D model on a computer system. The resulting model can be either completely image-based or it can be a combination of geometry and image data. Image-based rendering refers to methods of visualizing image-based models. The amount of data associated with such a model can be enormous. This significantly limits use of the model, since storage, data transmission, and rendering suffer because of the large size of the model. In addition, many images may be needed to display the object from different viewing angles.
In the past few years, there has been a proliferation of image-based rendering and modeling techniques. These techniques are popular because they provide a simple acquisition method and an accurate portrayal of the physical world. These techniques help to solve one of the biggest performance bottlenecks in 3D graphics, photo-realistic 3D content creation.
While geometry reconstruction using 3D photography techniques has been studied extensively and significant progress has been made in this area, reconstruction of the radiance function of scanned objects is a fairly new field with scarce tangible results reported so far.
In “Object Shape and Reflectance Modeling from Observation”, by Y.
Sato, M. D. Wheeler, and K. Ikeuchi, Proceedings of SIGGRAPH 97 pages 379–388, August 1997, the authors estimate a fixed reflectance model from a set of images of an object captured under controlled lighting conditions. The work of Sato, et al. assumes every point on the surface of the model has the same reflectance properties. The method of Sato, et al. works by separating the diffuse and specular reflection components from the color image sequence and then estimating the reflectance parameters of each reflection component separately. The proposed method of Sato, et al. is closely tied to the specific reflectance model specified. The proposed method works only for objects made of uniform material that can be approximated reasonably well with the specified reflectance model.
In “Inverse Global Illumination: Recovering Reflectance Models of Real Scenes From Photographs”, by Y. Yu, P. E. Debevec, J. Malik, and T. Hawkins, Proceedings of SIGGRAPH 99, pages 215–224, August 1999, the authors propose an approach that reconstructs the reflectance properties of a scene from a sparse set of photographs. Yu, et al. use an iterative optimization procedure that additionally allows the estimation of inter-reflections between surfaces in a scene. Yu, et al. call this procedure inverse global illumination. As with other known approaches, the method of Yu, et al. is limited to a predefined, low-parameter reflectance model that is not flexible enough to approximate complex surface material properties. The approaches of Sato, et al., and Yu, et al., are not robust because they require a heuristic procedure that separates the individual components of the reflectance model and estimates each one of the components independently.
Homomorphic factorization as described in “Homomorphic Factorization of BRDFs for High-Performance Rendering” by M. D. McCool, J. Ang, and A. Ahmad, Proceedings of SIGGRAPH 2001, pages 171–178, August 2001, generates a bi-directional reflectance distribution function (BRDF) factorization with positive factors only, which are easier and faster to render on current graphics hardware, and deals with scattered data without a separate re-sampling and interpolation algorithm. The algorithm of McCool, et al. deals with a special class of function factorizations                               f          ⁡                      (                                          x                1                            ,              …              ⁢                                                          ,                              x                n                                      )                          =                              ∏                          j              =              1                        n                    ⁢                                          ⁢                                    p              j                        ⁡                          (                              x                j                            )                                                          (                  Equation          ⁢                                          ⁢          1                )            by taking the logarithm of both sides of Equation (1) and solving the resulting linear data fitting problem                                           f            ~                    ⁡                      (                                          x                1                            ,              …              ⁢                                                          ,                              x                n                                      )                          =                              ∑                          j              =              1                        n                    ⁢                                          ⁢                                                    p                ~                            j                        ⁡                          (                              x                j                            )                                                          (                  Equation          ⁢                                          ⁢          2                )            Solving a linear system of equations takes O(N3) operations and requires O(N2) of intermediate storage, where N is the number of parameters being solved over, and the matrix is non-sparse. This non-sparsity is often the case in this formulation, where the radiance field is well sampled. In general, N will be on the order of 103 to 104, and therefore the method of McCool, et al. is fairly expensive both computationally and storage-wise. Additionally, homomorphic factorization can only handle a very narrow class of factorizations, which severely limits the generality of the method. Homomorphic factorization, which requires the inversion of a large matrix, is also not very conducive to parallelization and therefore cannot be mapped easily to streaming architectures.
Better techniques than are shown in the prior art are needed that make the representation of image-based models simple and more accurate. Such representations should be practical, compact, and easy to visualize on commodity graphics hardware.